Multimodal Approach toward Intelligent Video Production

This paper presents the framework of our intelligent video production, which has functions for multimodal data acquisition, accumulation to a database, and intelligent video editing. Since multimodal approach is most e ective and promising for semi-/automatic video production, we rst developed a prototype of multimodal acquisition system and database in which human behaviors are recorded in a multimodal way. By consulting it, we can intensively investigate the human behaviors in typical situations. We are developing algorithms for detecting typical human behaviors, for example, deictic movements. The behaviors are good keys to detecting the \focus", which means the most important portions to be recorded and to be presented to the audience. Thus, intelligent video production becomes possible by these intelligent processes. We brie y introduce, in this paper, our idea for detecting the focus of human behaviors in videotaping and emphasizing it.